Replicating ‘How Newspapers Reveal Political Power’

Adapted from Ban et al. (2019)

Sean Hambali

February 21, 2024

Introduction

Motivation

  • How to measure power?
    • Difficult to empirically observe power
  • Ban et al. (2019) provides an empirical measure of power via newspaper coverage
  • Main thesis: relative amount of space dedicated to a particular subject measures the relative power of that subject. Why?
  • Newsworthiness!
    • Size/impact of story matters
    • Prominence of actors also matters

Methods

What kind of power?

  • This paper specifically measures whether, and to what extent, political actors and offices possess the resources to influence political process.
    • Provides cardinal measure of power
    • Caveat 1: Newsworthiness does not necessarily translate to power, e.g. entertainment
    • Caveat 2: Potential bias from news outlets’ ideological preferences
    • Caveat 3: Does not measure the power of “shadowy” actors

Data

  • Primarily sourced from Newspapers.com
  • Stratified sample from 50% of archive
  • Focuses on years 1877-1977
  • Uses OCR tools and regex to count occurrences of political actors and offices in newspapers texts

(Validation) Results

Validation 1: Valuing congressional committees

  • Hypothesis: More powerful congressional committees should get more news coverage.
  • Ban et al. (2019) validates this hypothesis with committee “value” rankings constructed by Groseclose and Stewart (1998)

\[ \text{Relative Coverage of Committee}_i = \frac{\text{Committee}_i}{\sum^{19}_{i=1} \text{Committee}_i} \]

Result 1: Coverage vs Groseclose-Stewart rankings

Original

Replication

Validation 2: Newspaper coverage of House Speakers

Hypothesis: There should be spike in coverage in the periods where senators serve as party leaders

Result 2: Coverage of speakers before, during and after leadership

Original

Replication

Validation 3: Strong vs. weak mayors

  • Strong mayors (mayor-council):
    • Mayor and city council are elected separately
    • Mayor has broad executive authority
  • Weak mayors (council-manager):
    • City council is the only elected office
    • Council appoints city manager to oversee executive branch
    • Mayor has little to no executive authority
  • Hypothesis: Weaker mayor authority upon transition to council-manager setting, thus lower mentions in newspapers.

\[ \hat{Y}_{ijt} = \frac{N_{ijt}}{\sum^{J}_{j=1} N_{ijt}} \]

where \(j \in \{\text{Mayor}, \text{City Manager}, \text{City Council}\}\), \(N\), and \(\hat{Y}\) denote mentions and relative proportions, respectively.

Result 3A: Coverage of city offices, before and after reforms

Original

Replication

Result 3B: Effect of switching from Mayor-Council to Council-Manager

Original

Result 3B: Effect of switching from Mayor-Council to Council-Manager

Replication

Validation 4: Coverage of Massachussets Executive Council

  • Massachussets Executive Council is comprised of eight individuals elected from districts
  • In late 1964, the council was stripped of its statutory powers following corruption and bribery charges
  • Hypothesis: lower newspapers coverage on the Executive Council following the reform

\[ \text{EC}_t = \frac{\text{EC}_t}{\text{EC}_t + \text{G}_t} \]

where \(\text{EC}\) and \(\text{G}\) denote mentions of the Executive Council and the Governor in all available Massachussets newspapers, respectively.

Result 4: Coverage of Massachussets Executive Council

Original

Replication

Application: Measuring the power of state party committees

  • Prediction: State committees are no longer as powerful as they once were
  • Ban et al. (2019) uses newspaper coverage to measure the power of state committees relative to local committees

\[ \text{Relative Party Mentions}_{it} = \frac{\text{Party}_{it}}{\text{Party}_{it} + \text{Candidates}_{it}} \]

Result 5: Party Committee Power in 9 U.S. States

Original

Replication

A Closer Look at the Replication Exercise

Differences from the original results

  • Able to replicate all of the main findings
  • Minor differences in standard errors reported in regression table 3B

Replication diagnosis

  • The authors could not provide access to the raw newspapers data since they are proprietary data.
  • They only provide cleaned, aggregated term counts across years, states, etc.
  • As such, I wasn’t able to replicate the crucial first half of the paper, i.e., pre-processing the text.

Replication diagnosis

  • The authors used Stata to pre-process the term counts to generate measures of interest and R to visualize the output
  • I convert all Stata codes to R, posted all codes and output images/tables on GitHub.
  • I find one minor error in their code:
... %>%
  mutate(
    # Generate relative proportions
    rel_mayor_council_total = r_city_council + r_mayor,
    rel_mayor_council_control_total = r_control_sum_city_council + r_control_sum_mayor,
    rel_mayor_council = r_mayor / rel_mayor_council_total,
    rel_mayor_council_control = r_control_sum_mayor / rel_mayor_council_control_total, 
    
    # In the original paper, r_mayor was used instead of r_mayor_x
    rel_mayor_council_total_x = r_city_council_x + r_mayor_x, 
    rel_mayor_council_x = r_mayor_x / rel_mayor_council_total_x
  ) %>%
...

Possible extensions

  • Possible extensions:

    • Application to the Indonesian case: rapid decentralization in 1998 shifts political power from central government to local government
    • Possible extension to other mediums, e.g., news broadcast?
    • Robustness checks to the DiD regression under new approaches by Callaway & Sant’Anna (2020), Chaisemartin & D’Haultfœuille (2020)
  • Limitations for extension:

    • Limited nuance of the power being measured
    • Limited use in developing country context due to pervasive power of “shadowy” political actors
    • Limited use of newspapers in modern context, and possible changes to the incentive structure of newspaper outlets